' JP.2001 -1 1 6985.A [CLAIMS] 

t . 



1/1 V 



* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1]A camera which has an object recognition function, comprising: 

An object recognition means to recognize a photographic subject from a taken image. 

A sight line detecting means which detects a photography person's look. 

A picture logging means to set up a recognition position and a range of this object recognition 
means based on a detection result of this sight line detecting means. 

[Claim 2]A camera which has an object recognition function having a distance measurement 
means which measures a detection position of this sight line detecting means, or object distance 
of the neighborhood in the camera according to claim 1, starting this picture logging means 
based on a ranging result of this distance measurement means, and changing a range. 
[Claim 3]A camera comprising: 

An object recognition means to recognize a photographic subject from a taken image. 
An object recognition function characterized by using this sight line detecting means for the 
study input when it has a sight line detecting means which detects a photography person's look 
and this object recognition means needs study. 

[Claim 4]A camera using an object recognition function for at least 1 of AF of the camera 
concerned, AE, zoom, and a tailing function function in the camera according to any one of 
claims 1 to 3. 

[Claim 5]An object recognizing method characterized by comprising the following in a camera. 
Step A which detects a photography person's look. 

Step B which starts a recognition range of a photographic subject based on a detection result in 
this step A. 

Step C which performs object recognition processing in a range started by this step B. 

[Claim 6]An object recognizing method characterized by comprising the following in a camera. 
Step A which detects a photography person's look. 

Step B which performs study in object recognition processing using a detection result in this 
step A. 



[Translation done.] 
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DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention]This invention relates to the camera which recognizes a photographic 

subject from a taken image and performs necessary processing. 

[0002] 

[Description of the Prior Art]Generally, the object recognition art of recognizing the specific 
thing and person of a field is known. As the technique of extracting people's face from a taken 
image, beige data is extracted from an original image and the method of using as a face the 
cluster of the point judged to be a beige range measuring the strength of the light is known. They 
are JP,52-156624,A, JP,53-14521,A, JP,53-145622,A, etc. Furthermore by JP,4-346333,A t the 
method of judging a face area in changing light measurement data into hue and chroma 
saturation, and creating and analyzing this two-dimensional histogram is indicated. 
[0003]In JP,8-063597,A, the face-candidates field equivalent to the shape of people's face is 
extracted, and a face area is determined from the characteristic quantity in the field. An option 
extracts the outline of people's face from a picture, and determines a face area. In an option, 
preparing the template which carried out shape of two or more faces, calculating correlation with 
that template and picture, and considering it as a face-candidates field with this correlation 
value extracts people's face. Furthermore by an option, it becomes possible to repeat study 
using a neural network and to recognize the thing and face of a photographic subject. The model 
in which this used Neocognitoron is known. (Fukushima: Neural circuit model Neocognitoron 
[ which is not influenced by position gap / of a pattern recognition function ], Institute of 
Electronics and Communication Engineers paper magazine A, and J62-A(10) PP658-665, 
Oct 1979). About a neural network's learning method, there is "1 examination (an Institute of 
Television Engineers of Japan technical report, Vol.14, No50-1 990.9, 7-12) of face picture 
discernment using a neural network" etc. 

[0004]Art and reference works are introduced to JP,9-251534,A or JP,10-232934,A in detail as 

known art about facial recognition, face detection, and eye detection. 

[0005] 

[Problem(s) to be Solved by the Invention]In the above-mentioned conventional example, in 
order to recognize a photographic subject from an image pick, image processing had to be 
carried out about the whole image pick, calculation of a huge quantity was needed, and even if it 
was highly efficient CPU and an arithmetic unit, big time was needed. 

[0006]This invention is a thing of such a situation, and was made. and an object of this invention 
is to provide a high speed, the camera which has an object recognition function in which it is 
highly precise and object recognition can be realized, and an object recognizing method. 
[0007] 

[Means for Solving the Problem]In order to attain said purpose, a camera which has an object 
recognition function is constituted as following (1) - (4), and an object recognizing method 
consists of this inventions in a passage of the following (5) and (6). 

[0008](D A camera which has the object recognition function provided with an object recognition 
means to recognize a photographic subject from a taken image, a sight line detecting means 
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which detects a photography person's look, and a picture logging means to set up a recognition 
position and a range of this object recognition means based on a detection result of this sight 
line detecting means. 

[0009](2) A camera which has the object recognition function to have a distance measurement 
means which measures a detection position of this sight line detecting means, or object distance 
of the neighborhood in a camera of the aforementioned (1) statement, to start this picture 
logging means based on a ranging result of this distance measurement means, and to change a 
range. 

[0010](3) A camera which has the object recognition function to use this sight line detecting 
means for the study input when it has an object recognition means to recognize a photographic 
subject from a taken image, and a sight line detecting means which detects a photography 
person's look and this object recognition means needs study. 

[0011](4) The above (1) Camera which uses an object recognition function for at least 1 of AF of 
the camera concerned, AE, zoom, and a tailing function function in a camera of a statement at 
either of - (3). 

[0012](5) Step A which is an object recognizing method in a camera and detects a photography 
person's look. An object recognizing method provided with Step B which starts a recognition 
range of a photographic subject based on a detection result in this step A, and Step C which 
performs object recognition processing in a range started by this step B. 

[0013](6) An object recognizing method which is an object recognizing method in a camera and 
was provided with Step A which detects a photography person's look, and Step B which 
performs study in object recognition processing using a detection result in this step A. 
[0014] 

[Embodiment of the Invention]The example of 1 eye REFUKAMERA explains an embodiment of 
the invention in detail below. This invention can be similarly carried out in the form of not only a 
film camera like an example but a still video camera (digital camera) and a video camera. It is 
supported by explanation of an example and can also carry out in the form of an object 
recognizing method. 
[0015] 

[Example](Example 1) Drawing 1 is a block diagram showing the composition of "1 Eye 
REFUKAMERA" which is Example 1. 

[0016]In drawing 1 , a taking lens and 102 101 A quick return mirror, 103 a focusing plate and 104 
a pentaprism and 105 Light dividing prism, 106 an eyepiece and 107 the area sensor for ranging, 
and 108 The area sensor for looks, 109 — a film plane and 110 — as for an object recognition 
part and 114, the image pick-up treating part of an area sensor and 112 are [ a distance primary 
detecting element and 116 ] line of sight detection parts an outputting part and 115 a picture 
logging treating part and 113 a shutter curtain and 111. 

[0017]An object image bends a photographing optical axis 90 degrees by the quick return mirror 
102 through the taking lens 101, and primary image formation is carried out with the focusing 
plate 103. The focusing plate 103 consists of the mat and field lens which are located in the 
primary image formation face of the above-mentioned taking lens 101. An optical path change is 
carried out to finders by the pentaprism 104, it separates into the area sensor 107 for ranging, 
and the eyepiece 106 via the light dividing prism 105, respectively, and the picture of the 
focusing plate 103 is observed by the photography person 117. Image formation of the observers 
117 ophthalmic image js carried out to the area sensor 108 for looks via the eyepiece 106 and 
the light dividing prism 105. 

[0018]Two taken images in which the pupil posions by which image formation was carried out to 
the area sensor 107 for ranging, and which are mentioned later differ supply the image data 
which the image pick-up treating part 1 1 1 memorizes, and is needed for the picture logging 
treating part 112 and the distance primary detecting element 1 1 5 if needed, respectively. In the 
picture logging treating part 112, image data is outputted for the picture of a position and a size 
to the selection logging object recognition part 113 from a taken image based on the distance 
information of the distance primary detecting element 115, and the line of sight position 
information on the line of sight detection part 116. The logging position at this time is decided 
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from the line of sight positron information in the field of the photography person by the line of 
sight detection part 116. The size of a picture is decided based on the size information suitable 
for the object photographic subject which is going to carry out prediction arithmetic of the 
actual dimension of a photographic subject, and is going to recognize it by the distance 
information of the distance primary detecting element 115 set up beforehand. Let the detection 
position in the photographic subject of the distance primary detecting element 115 at this time 
be that position or its near position of a taken image based on the information from the line of 
sight detection part 116. The object recognition part 113 recognizes a photographic subject from 
the picture from the picture logging treating part 112, and is outputted by the outputting part 
114. The object recognition part 113 should just carry out recognition processing only of the 
imaging range specified based on the picture information from the picture logging treating part 
1 1 2, and it becomes possible to process at high speed by pressing down an operation to 
minimum. On the other hand, the distance primary detecting element 115 ranges the object 
distance of the position or the neighborhood based on the line of sight position information from 
the line of sight detection part 116, and outputs distance information to the above-mentioned 
picture logging treating part 112. The line of sight detection part 116 detects a line of sight 
position from the ophthalmic image from the area sensor 108 for looks, and outputs it to the 
above-mentioned distance primary detecting element 115 and the picture logging treating part 
1 12 as line of sight position information. 

[0019]When taking a photograph on a film, the quick return mirror 102 and the shutter 110 are 
evacuated from a photographing light bunch, and it exposes on the film 109. 
[0020]It becomes possible to become data processing of only picture information required for 
origin, and to process the area information which object recognition processing chose by the 
look at high speed by a series of the above composition and operations. It also becomes possible 
to perform complicated object recognition processing more advanced than the part which 
became high-speed. Since it is chosen by the look, it becomes possible to recognize with high 
accuracy. 

[0021]It explains using drawing 2 still in detail about the circumference of the area sensor 107 
for ranging, and the area sensor 108 for looks. 

[0022]105 is light dividing prism which has the semi transmission mirror part 105a. This prism 
105 made the part penetrate so that it may be visible to the photography person 1 17 in a finder 
light bunch, and is doubled and provided with the operation which leads the remaining light flux to 
the area sensor 107 for ranging, and the operation which leads a photography person's eyeball 
image to the area sensor 108 for looks. The mirror for optical-path conversion and 203 are the 
two-dimensional image formation lenses for ranging a phase difference system, and 202 carries 
out image formation of the image pick-up field on the area sensor 107 for ranging. 204a and 204b 
are the lighting of a photography person's eyeball 1 17, and are arranged near the eyepiece 106. 
201 is an image formation lens for carrying out image formation of a photography person's 
eyeball to the area sensor 108 for looks. With the lighting 204a and 204b, it reflects by the 
translucent mirror 105a in the light dividing prism 105, image formation of the photography 
person's 117 ophthalmic image is carried out with the image formation lens 201 via the eyepiece 
106, and image formation is carried out on the area sensor 108 for looks. 
[0023]Ranging operation is further explained using drawing 3 . Drawing 3 is equivalent to the 
abstract figure about the optical system in connection with the distance detection in drawing 1 
and drawing 2 . Re-image formation of the picture by which primary image formation was carried 
out to the mat face of the focusing plate 103 is carried out to 107a and 107b of the area sensor 
107 for ranging, respectively with the field lens and the secondary image formation lens 203 of 
the focusing plate 103. Light flux is drawn from the pupil posion which changes in 107a and 107b 
with throttle plates 301, respectively at this time. The imaging screens 107a and 107b which had 
azimuth difference in predetermined by this composition are obtained. An imaging screen with 
this azimuth difference is divided into Brock of a mxn individual (m=n may be sufficient), 
respectively, and if publicly known correlation operation of the signal in Brock who faces, 
respectively is performed, the distance and the defocusing amount to the object in a previous 
block can be measured by the principle of triangulation. This distance and the relation of 
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defocusing turn into a norrlinear relation with the focal distance of that lens, a focal position, and 
the characteristic peculiar to a lens. The means which carries out correction conversion of these 
relations if needed is publicly known. 

[0024]About the camera which has this composition, it is indicated in detail in the Japanese- 
Patent-Application-No. No. 278433 [ five to ] gazette etc. 

[0025]Next, the sight line detecting means a camera recognizes it to be which position of the 
finder image plane of a camera, i.e., the focusing plate up, the photography person who looks into 
the finder of a camera is looking at is explained. 

[0026]The device (for example, eye camera) which detects which position on an observed face 
the observer is observing conventionally and which detects what is called a look (optical axis) is 
provided variously. As a method of detecting the look, for example in JP,1-274736,A, the parallel 
pencil from a light source is projected to the anterior eye segment of an observers eyeball, and 
it is asking for the optical axis using the image formation position of the corneal reflex image by 
the catoptric light from a cornea, and a pupil. 
[0027]Next, it explains further using a flow chart 

[0028]The flow chart is shown in drawing 4 . A photography person's look is first detected at 
Step 401 (at drawing 4 , it is the same as that of S401, a brief sketch, and the following), and it 
progresses to Step 402 in quest of the line of sight position of a field. At Step 402, the field 
image of the periphery is captured based on the line of sight position information searched for at 
Step 401, and it progresses to Step 403. At Step 403, object recognition processing is performed 
from the field picture captured at Step 402, a photographic subject is detected, and it 
progresses to Step 404. At Step 404, the photographic subject detection result obtained at Step 
403 is outputted. 

[0029]Suppose that it is possible to recognize the photographic subject of a line of sight position 
efficiently by a series of above flows. 

[0030] Drawing 5 explains the contents of processing of Step 402 of drawing 4 in more detail. At 
Step 501, ranging in the line of sight position is performed based on a photography person's line 
of sight position information acquired at Step 401 of drawing 4 , and it progresses to Step 502. 
The conventional method mentioned above may be sufficient as this ranging method. At Step 
502, it progresses to Step 503 in quest of the photographing magnification information on the 
distance with terms and conditions, such as a taking lens and an area sensor, to the publicly 
known technique from the distance information of a line of sight position. At Step 503, it 
progresses to Step 504 in quest of the image range corresponding to the actual dimension of the 
subject to recognize based on the photographing magnification information on Step 502. This 
Tokizane size information is still better to prepare beforehand and to also set up beforehand the 
relation of that size information and recognition processing range of a picture. The work which 
prevents the logging mistake by the detection error of a line of sight position by setting up a 
treating range more greatly somewhat by this can be given. At Step 504, the field picture for 
object recognition is photographed based on the treating range information on the picture set up 
at Step 503, and the lump drawing 4 step 403 performs an object recognition place. Object 
recognition becomes realizable good from the field picture of a line of sight position by the above 
flow. 

[0031] Drawing 6 explains an image of operation further. Three figures on the left-hand side of 
drawing 6 are frames about the position by which line of sight detection was carried out to the 
image pick, and show the distance measurement data in the position numerically. Three right- 
hand side figures have shown the object recognition treating range called for based on each of 
the line of sight position and distance information by the frame. Casel shows a 3-m ranging 
result by the case where Case3 chooses a right person for a 4-m ranging result by the case 
where Case2 chooses a left person for a 5-m ranging result by the case where the person at the 
very back of central is chosen. Since the real distances of a photographic subject differ by each 
Case, the image ranges for recognition processing also differ, respectively, and it becomes so 
small that a selection range is so large that it is near though natural and it is far. 
[0032]The output of object recognition can be applied to various functions, such as AF 
(automatic-focusing regulation), AE (automatic exposure adjustment), zoom, and tailing, and the 
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easier more high definition photography of it is attained. 

[0033](Example 2) Operation of the "camera" which is Example 2 is explained using a flow chart 

[0034]For example, this example needs study for an object recognition algorithm, it explains the 
example which used the look as an input of that study in the case of neuro etc. 
[0035]The flow chart is shown in drawing 7 . At Step 701, the field which is going to carry out 
object recognition is incorporated and it progresses to Step 702. At Step 702, he performs 
object recognition processing and follows to Step 703 the picture captured at Step 701. At Step 
703, the recognition processing result in Step 702 is judged, and if it is O.K., and it is NG, it will 
progress to an end to Step 704. At this time, the recognition processing itself may judge O.K. 
and NG automatically, and a photography person etc. may operate them. At Step 704, object 
recognition processing is set as learning mode, and it progresses to Step 705. At Step 705, the 
correct answer of an object recognition processing result is inputted using a look, and it 
progresses to Step 706. At this time, the input method in a look may input a correct answer, 
looking at a taken image, and may input it in a sign, a correct answer position, etc. In Step 706, 
the correct answer inputted at Step 705 performs learning processing of object recognition 
processing, and improvement in recognition precision is aimed at and it ends. 
[0036]It becomes possible by it coming out to perform the study input's of object recognition 
processing easily, becoming possible, and repeating by the above flow, to recognize to high 
degree of accuracy more. 
[0037] 

[Effect of the Invention]As explained above, according to this invention, a high speed and the 
advanced processing can carry out by being highly precise and using the object recognition 
processing result become easy about object recognition by using a look input As a result a 
quality picture can be photoed automatically and easily. 



[Translation done.] 
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and a large quantity of calculations were needed. 
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detecting means, and the position of an object subject is 
specified. A range-finding means is provided which measure 
the detection position of the line of sight detecting means 
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